LLM-BASED AGENTS

LLM-Based Agent Society Investigation: Collaboration and Confrontation in Avalon Gameplay (2024.emnlp-main)

Challenge:	Existing web agents relying on supervised fine-tuning struggle with generalization and robustness due to insufficient reasoning capabilities when handling the inherently dynamic nature of web interactions.
Approach:	They propose a large language model-empowered web agent that trains using a rule-based reinforcement learning framework to enhance single-step reasoning and planning for business-oriented web navigation tasks.
Outcome:	The proposed agent outperforms baseline LLM-based agents on the WorkArena benchmark by 10.26–16.59%.

Copied to clipboard

Yihuai Lan, Zhiqiang Hu, Lei Wang, Yang Wang, Deheng Ye, Peilin Zhao, Ee-Peng Lim, Hui Xiong, Hao Wang

Challenge:	Existing studies on LLM agents' social behaviors are lacking . previous studies focused on positive social behaviors, leaving research on negative social behaviors relatively scarce.
Approach:	They propose a framework that features a multi-agent system facilitating efficient communication and interaction with LLM agents.
Outcome:	The proposed framework is based on Avalon and evaluates on game success and analyzes agents’ social behaviors.

I-MCTS: Enhancing Agentic AutoML via Introspective Monte Carlo Tree Search (2026.findings-eacl)

Copied to clipboard

Zujie Liang, Feng Wei, Wujiang Xu, Yuxi Qian, Lin Chen, Xinhui Wu

Preemptive Detection and Correction of Misaligned Actions in LLM Agents (2025.emnlp-main)

Challenge:	Existing LLM-based agents struggle with low diversity and suboptimal code generation.
Approach:	They propose an approach that iteratively expands tree nodes through an introspective process that meticulously analyzes solutions and results from parent and sibling nodes.
Outcome:	The proposed approach shows a 4% improvement in performance compared to the strong open-source AutoML agents.

Copied to clipboard

Haishuo Fang, Xiaodan Zhu, Iryna Gurevych

Challenge:	Large Language Models (LLMs) have revolutionized human-AI collaboration by enabling autonomous agents to execute complex, multi-step tasks.
Approach:	They propose a method that leverages the belief reasoning ability of LLMs to detect misaligned actions.
Outcome:	Experiments on three widely used tasks show that InferAct outperforms other methods on Marco-F1 and emnlp2025.

Theory of Mind for Multi-Agent Collaboration via Large Language Models (2023.emnlp-main)

Copied to clipboard

Huao Li, Yu Chong, Simon Stepputtis, Joseph Campbell, Dana Hughes, Charles Lewis, Katia Sycara

HotelQuEST: Balancing Quality and Efficiency in Agentic Search (2026.eacl-industry)

Challenge:	Recent large language models (LLMs) have demonstrated impressive accomplishments in reasoning and planning, but their abilities in multi-agent collaborations remain unexplored.
Approach:	They propose to use explicit belief state representations to enhance task performance and the accuracy of ToM inferences for LLM-based agents.
Outcome:	The proposed model improves performance and accuracy of ToM inferences for LLM-based agents.

Copied to clipboard

Guy Hadad, Shadi Iskander, Sofia Tolmach, Oren Kalinsky, Haggai Roitman, Ran Levy

Green & Sustainable NLP

Challenge:	Existing benchmarks for agentic search focus primarily on answer quality, overlooking efficiency factors that are critical for real-world deployment.
Approach:	They propose a benchmark for hotel search queries that includes 214 hotel query queries that range from simple factual requests to complex queries.
Outcome:	The proposed benchmarks show that LLM-based agents achieve higher accuracy than traditional retrievers, but at substantially higher costs due to redundant tool calls and suboptimal routing that fails to match query complexity to model capability.

Systematic Biases in LLM Simulations of Debates (2024.emnlp-main)

Copied to clipboard

Amir Taubenfeld, Yaniv Dover, Roi Reichart, Ariel Goldstein

Challenge:	Current research suggests that LLM-based agents become increasingly human-like in their performance, sparking interest in using these AI agents as substitutes for human participants in behavioral studies.
Approach:	They propose to use LLMs to simulate political debates on topics that are important aspects of people’s day-to-day lives and decision-making processes.
Outcome:	The proposed model can simulate political debates on topics that are important aspects of people’s day-to-day lives and decision-making processes.

DialogGuard: Multi-Agent Psychosocial Safety Evaluation Interface of Sensitive LLM Responses (2026.acl-demo)

Copied to clipboard

Han Luo, Guy Laban

Dialogue Systems & Conversational Agents

Challenge:	Existing tools do not surface subtler psychosocial harms, nor provide explainable rationales that practitioners need.
Approach:	They propose an open-source system that lets practitioners inspect, stress-test, and create audit trails for prompted LLM agents across five psychosocial safety dimensions.
Outcome:	The open-source DialogGuard system lets practitioners inspect, stress-test, and create audit trails for prompted LLM agents across five psychosocial safety dimensions.

TPTU-v2: Boosting Task Planning and Tool Usage of Large Language Model-based Agents in Real-world Industry Systems (2024.emnlp-industry)

Copied to clipboard

Yilun Kong, Jingqing Ruan, YiHong Chen, Bin Zhang, Tianpeng Bao, Shi Shiwei, Du Qing, Xiaoru Hu, Hangyu Mao, Ziyue Li, Xingyu Zeng, Rui Zhao, Xueqian Wang

Agent-Ops: A Multi-Agent Orchestration Framework for End-to-End SOP Automation in E-Commerce Operations (2026.acl-industry)

Challenge:	Large language models have demonstrated proficiency in addressing tasks that necessitate a combination of task planning and the usage of external tools.
Approach:	They propose a framework to enhance the task planning and tool usage abilities of LLMs in industrial systems.
Outcome:	The proposed framework enhances the task planning and tool usage abilities of LLM-based agents in industrial systems.

Copied to clipboard

Apoorva Singh, Sanjay Agrawal, Sayanta Adhikari, Vinayak S Puranik, Shivam Tiwari, Dheeraj Assudani

Challenge:	Existing Large Language Models fail to execute multistep operational workflows requiring precise procedural adherence.
Approach:	They propose an end-to-end multi-agent framework automating Standard Operating Procedures in e-commerce.
Outcome:	The proposed framework achieves 85-97% accuracy and a 94.2% execution consistency in e-commerce . it is based on a human-AI framework that transforms ambiguous documentation into automation-ready specifications .

CASPER: Bridging Discrete and Continuous Prompt Optimization through Feedback-Guided Gradient Descent (2026.eacl-industry)

Copied to clipboard

Aryan Jain, Pushpendu Ghosh, Promod Yenigalla

SynWorld: Virtual Scenario Synthesis for Agentic Action Knowledge Refinement (2025.acl-short)

Challenge:	Existing pipelines for generative tasks require extensive manual effort and domain expertise to achieve task-optimal performance.
Approach:	They propose a framework bridging discrete and continuous prompt optimization through feedback-guided gradient descent in embedding space.
Outcome:	The proposed framework bridges discrete and continuous prompt optimization through feedback-guided gradient descent in embedding space.

Copied to clipboard

Runnan Fang, Xiaobin Wang, Yuan Liang, Shuofei Qiao, Jialong Wu, Zekun Xi, Ningyu Zhang, Yong Jiang, Pengjun Xie, Fei Huang, Huajun Chen

Challenge:	Using Large Language Models (LLMs)-based agents can enhance their understanding of environments and tasks.
Approach:	They propose a framework that allows agents to synthesize possible scenarios with multi-step action invocation within the action space and perform Monte Carlo Tree Search exploration to refine their action knowledge in the current environment.
Outcome:	The proposed framework synthesizes possible scenarios with multi-step action invocation within the action space and performs Monte Carlo Tree Search exploration to refine action knowledge in the current environment.

Preference-Aware Memory Update for Long-Term LLM Agents (2026.findings-acl)

Copied to clipboard

Haoran Sun, Zekun Zhang, Shaoning Zeng

Dialogue Systems & Conversational Agents

Challenge:	Existing methods for integrating long-term memory do not provide dynamic and personalized memory refinement.
Approach:	They propose a long-term memory update mechanism that enables dynamic and personalized memory refinement.
Outcome:	The proposed mechanism improves the performance of LLM-based agents in five tasks.

LLM-Based Explicit Models of Opponents for Multi-Agent Games (2025.naacl-long)

Copied to clipboard

XiaoPeng Yu, Wanpeng Zhang, Zongqing Lu

Challenge:	Existing approaches to model adversarial and cooperative interactions often focus on treating other agents as separate entities with their own intentions and strategies.
Approach:	They propose a model of opponents based on Large Language Models (LLMs) that constructs an individual model for each opponent and aligns these models working in synergy through a bi-level feedback-refinement framework.
Outcome:	The proposed model outperforms single-model approaches in multi-player deduction games, showing that it significantly enhances agents’ decision-making.

EASYTOOL: Enhancing LLM-based Agents with Concise Tool Instruction (2025.naacl-long)

Copied to clipboard

Siyu Yuan, Kaitao Song, Jiangjie Chen, Xu Tan, Yongliang Shen, Kan Ren, Dongsheng Li, Deqing Yang

Neuro-Symbolic Agentic Reinforcement Learning for Long-Term Original Character Companionship and Interaction (2026.acl-short)

Challenge:	EASYTOOL combines tools from diverse tool documentation into a single tool instruction.
Approach:	They propose a framework that transforms tool documentation into a unified tool instruction.
Outcome:	EASYTOOL combines extensive tool documentation into a concise tool instruction . it reduces token consumption and improves performance of LLM-based agents .

Copied to clipboard

Zhenhan Huang

Knowledge Representation

Challenge:	Existing LLM-based agents that are optimized by prompting or supervised fine-tuning exhibit a generalization gap in long-horizon, socially rich interactions.
Approach:	They propose a framework that formalizes OC companion agents’ interactions as a POMDP and decomposes the agent into three sub-policies optimized via closed-loop RL from AI feedback with verifiable rewards in a graph-constrained action space.
Outcome:	The proposed framework formalizes OC companion agents’ interactions as a POMDP and decomposes the agent into three sub-policies (Router, Memory, and Persona) with verifiable rewards in a graph-constrained action space.

PExA: Parallel Exploration Agent for Complex Text-to-SQL (2026.acl-short)

Copied to clipboard

Tanmay Parekh, Ella Hofmann-Coyle, Shuyi Wang, Sachith Sri Ram Kothur, Srivas Prasad, Yunmo Chen

Programming Languages in NLP

Challenge:	Recent work in text-to-SQL has explored toolaugmented LLMs, deep planning, and agentic workflows to address complex challenges.
Approach:	They validated a framework for text-to-SQL, Spider 2.0, with 70.2% execution accuracy.
Outcome:	The proposed framework achieves 70.2% execution accuracy on a state-of-the-art benchmark for text-to-SQL, Spider 2.0.

BASES: Large-scale Web Search User Simulation with Large Language Model based Agents (2024.findings-emnlp)

Copied to clipboard

Ruiyang Ren, Peng Qiu, Yingqi Qu, Jing Liu, Xin Zhao, Hua Wu, Ji-Rong Wen, Haifeng Wang

ChemAmp: Amplified Chemistry Tools via Composable Agents (2026.findings-acl)

Challenge:	Existing research on web search rely on real-user experiments, which can be costly to scale up.
Approach:	They propose a user simulation framework with LLM-based agents that can generate unique user profiles at scale.
Outcome:	The proposed framework can generate unique user profiles at scale, leading to diverse search behaviors.

Copied to clipboard

Zhucong Li, Powei Chang, Jin Xiao, Zhijian Zhou, Qianyu He, Jiaqing Liang, Fenglei Cao, Xu Yinghui, Yuan Qi

Challenge:	LLM-based agents are powerful tools for automating complex scientific workflows, especially in chemistry, but their single-task performance is limited by tool constraints.
Approach:	They propose a framework that optimizes the collective capabilities of specialized tools by dynamic coordination within individual tasks.
Outcome:	The proposed framework outperforms chemistry-specialized models, generalist LLMs, and agent systems with tool orchestration.

Large Language Model-based Human-Agent Collaboration for Complex Task Solving (2024.findings-emnlp)

Copied to clipboard

Xueyang Feng, Zhi-Yuan Chen, Yujia Qin, Yankai Lin, Xu Chen, Zhiyuan Liu, Ji-Rong Wen

Demonstrating ViviDoc: Generating Interactive Documents through Human-Agent Collaboration (2026.acl-demo)

Challenge:	Recent advances in large language models have led to the development of LLM-based autonomous agents.
Approach:	They propose a Reinforcement Learning-based Human-Agent Collaboration method which trains a policy model to determine the most opportune stages for human intervention within the task-solving process.
Outcome:	The proposed method improves human-agent collaboration significantly through well-planned, limited human intervention.

Copied to clipboard

Yinghao Tang, Yupeng Xie, Yingchaojie Feng, Tingfeng Lan, Wei Chen

Challenge:	Recent LLM-based agents can automate content creation, but naively applying them yields uncontrollable and unverifiable outputs.
Approach:	They propose a human-agent collaborative system that generates interactive educational documents from a single topic input.
Outcome:	The proposed system generates documents comparable in quality to human-authored ones.

RTADev: Intention Aligned Multi-Agent Framework for Software Development (2025.findings-acl)

Copied to clipboard

Jie Liu, Guohua Wang, Ronghui Yang, Jiajie Zeng, Mengchen Zhao, Yi Cai

AgentFactory: A Self-Evolving Framework Through Executable Subagent Accumulation and Reuse (2026.acl-demo)

Challenge:	Large Language Models (LLMs) are efficient assistants to humans in software development tasks, but they can cause errors during the development process.
Approach:	They propose an intention aligned multi-agent framework that ensures that all agents work based on a consensus.
Outcome:	The proposed framework reduces errors and improves the quality of generated software code.

Copied to clipboard

Zhang Zhang, Shuqi Lu, Hongjin Qian, Di He, Zheng Liu

Athena: Safe Autonomous Agents with Verbal Contrastive Learning (2024.emnlp-industry)

Challenge:	Existing frameworks for building LLM-based agents treat agent behavior as static-knowledge gained during execution is not preserved for future use.
Approach:	They propose a new paradigm that preserves successful task solutions as executable subagent code rather than textual experience.
Outcome:	The proposed agent-based agent-driven paradigm preserves successful tasks as executable subagent code rather than textual experience.

Copied to clipboard

Tanmana Sadhu, Ali Pesaranghader, Yanan Chen, Dong Yi

FinHEAR: Human Expertise and Adaptive Risk-Aware Temporal Reasoning for Financial Decision-Making (2025.findings-emnlp)

Challenge:	Existing safety benchmarks on the ability of large language models to perform tasks are lacking.
Approach:	They propose a framework that leverages verbal contrastive learning to guide agents towards safety . they use past safe and unsafe trajectories as in-context examples to guide them towards safety.
Outcome:	The proposed framework leverages verbal contrastive learning to guide agents towards safety while performing tasks.

Copied to clipboard

Jiaxiang Chen, Mingxi Zou, Zhuo Wang, Qifan Wang, Danny Dongning Sun, Zhang Chi, Zenglin Xu

Text2Mem: A Unified Memory Operation Language for Memory Operating System (2026.findings-acl)

Challenge:	Large language models often overlook key behavioral patterns underlying human financial behavior.
Approach:	FinHEAR is a multi-agent framework for human expertise and Adaptive Risk-aware reasoning.
Outcome:	FinHEAR outperforms baseline models in trend forecasting and decision-making.

Copied to clipboard

Leo Wang, Lihai Yang, Boyu Chen, Kerun Xu, Gongyi Zou, Bo Tang, Feiyu Xiong, Siheng Chen, Zhiyu li

Challenge:	Existing memory frameworks lack a formal, executable specification for memory control.
Approach:	They propose a unified memory operation language that standardizes translation of natural-language instructions into reliable execution.
Outcome:	The proposed language standardizes translation of natural-language instructions into reliable execution.

Triad: A Framework Leveraging a Multi-Role LLM-based Agent to Solve Knowledge Base Question Answering (2024.emnlp-main)

Copied to clipboard

Chang Zong, Yuchen Yan, Weiming Lu, Jian Shao, Yongfeng Huang, Heng Chang, Yueting Zhuang

Natural Language Interfaces

Challenge:	Recent advances with LLMs have shown promising results across various tasks, but their use in answering questions from knowledge bases remains largely unexplored.
Approach:	They propose a framework that utilizes an LLM-based agent with multiple roles for KBQA tasks.
Outcome:	The proposed framework outperforms state-of-the-art systems on the LC-QuAD and YAGO-QA benchmarks yielding F1 scores of 11.8% and 20.7%, respectively.

CompileAgent: Automated Real-World Repo-Level Compilation with Tool-Integrated LLM-based Agent System (2025.acl-long)

Copied to clipboard

Li Hu, Guoqiang Chen, Xiuwei Shang, Shaoyin Cheng, Benlong Wu, LiGangyang LiGangyang, Xu Zhu, Weiming Zhang, Nenghai Yu

Challenge:	CompileAgent is the first LLM-based agent framework dedicated to repo-level compilation.
Approach:	They propose a LLM-based agent framework dedicated to repo-level compilation.
Outcome:	The proposed method significantly improves compilation success rate, ranging from 10% to 71%.

INVESTORBENCH: A Benchmark for Financial Decision-Making Tasks with LLM-based Agent (2025.acl-long)

Copied to clipboard

Haohang Li, Yupeng Cao, Yangyang Yu, Shashidhar Reddy Javaji, Zhiyang Deng, Yueru He, Yuechen Jiang, Zining Zhu, K.p. Subbalakshmi, Jimin Huang, Lingfei Qian, Xueqing Peng, Jordan W. Suchow, Qianqian Xie

AndroidGen: Building an Android Language Agent under Data Scarcity (2025.acl-long)

Challenge:	Recent advances have underscored the potential of large language model (LLM)-based agents in financial decision-making.
Approach:	They propose to evaluate LLM agents using 13 different LLMs as backbone models across various market environments and tasks.
Outcome:	The proposed framework assesses the reasoning and decision-making capabilities of 13 different LLMs across various market environments and tasks.

Copied to clipboard

Hanyu Lai, Junjie Gao, Xiao Liu, Yifan Xu, Shudan Zhang, Yuxiao Dong, Jie Tang

DemonAgent: Dynamically Encrypted Multi-Backdoor Implantation Attack on LLM-based Agent (2025.findings-emnlp)

Challenge:	Existing LLMs lack high-quality data sources and lack robust data filtration strategies.
Approach:	They develop a framework to enhance the capabilities of LLM-based agents under data scarcity.
Outcome:	The proposed framework improves the capabilities of LLM-based agents under data scarcity.

Copied to clipboard

Pengyu Zhu, Zhenhong Zhou, Yuanhe Zhang, Shilinlu Yan, Kun Wang, Sen Su

LLM-Based Offline Learning for Embodied Agents via Consistency-Guided Reward Ensemble (2024.findings-emnlp)

Challenge:	a new method for detecting advanced backdoors is proposed to bypass safety audits.
Approach:	They propose a backdoor implantation strategy that introduces dynamic encryption to bypass safety audits.
Outcome:	The proposed method achieves an attack success rate approaching 100% while maintaining a detection rate of 0%.

Copied to clipboard

Yujeong Lee, Sangwoo Shin, Wei-Jin Park, Honguk Woo

Challenge:	Employing large language models (LLMs) to enable embodied agents has become popular, yet it presents several limitations in practice.
Approach:	They propose a consistency-guided reward ensemble framework to train agents offline via offline reinforcement learning (RL) they use spatio-temporally consistent rewards to derive domain-grounded rewards from training datasets.
Outcome:	The proposed framework outperforms state-of-the-art LLM-based agents with 8B parameters and has 117M parameters for agent policy network and only for training.

AutoPlan: Automatic Planning of Interactive Decision-Making Tasks With Large Language Models (2023.findings-emnlp)

Copied to clipboard

Siqi Ouyang, Lei Li

MPO: Boosting LLM Agents with Meta Plan Optimization (2025.findings-emnlp)

Challenge:	Existing methods for making decisions in grounded environments require costly gradient computation or lengthy in-context demonstrations.
Approach:	They propose an approach to guide LLM-based agents to accomplish interactive decision-making tasks by using an LLM prompt and a task-solving plan.
Outcome:	The proposed approach outperforms human-written demonstrations on ALFWorld and HotpotQA by 8%.

Copied to clipboard

Weimin Xiong, Yifan Song, Qingxiu Dong, Bingchan Zhao, Feifan Song, null XWang, Sujian Li

COLA: Collaborative Multi-Agent Framework with Dynamic Task Scheduling for GUI Automation (2025.emnlp-main)

Challenge:	Existing methods for interactive planning tasks suffer from planning hallucinations and require retraining for each new agent.
Approach:	They propose a framework that leverages explicit guidance through meta plans to assist agent planning and enables continuous optimization based on feedback from the agent’s task execution.
Outcome:	The proposed framework outperforms existing baselines on two representative tasks and significantly improves task completion efficiency and generalization capabilities.

Copied to clipboard

Di Zhao, Longhui Ma, Siwei Wang, Miao Wang, Zhao Lv

Challenge:	Existing methods for implementing LLMs are limited by their complexity and lack fault tolerance mechanism.
Approach:	They propose a scenario-aware agent Task Scheduler that decomposes task requirements into atomic capability units and dynamically selects the optimal agent from a decision agent pool.
Outcome:	The proposed framework achieves competitive performance among GUI Agent methods with an average accuracy of 31.89% on the GAIA dataset.

PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traits (2024.findings-naacl)

Copied to clipboard

Hang Jiang, Xiajie Zhang, Xubo Cao, Cynthia Breazeal, Deb Roy, Jad Kabbara

MAPRO: Recasting Multi-Agent Prompt Optimization as Maximum a Posteriori Inference (2026.findings-eacl)

Challenge:	Recent studies have shown that LLMs can generate content that aligns with their assigned personality traits, but there is limited research on whether they consistently reflect specific personality traits.
Approach:	They propose to study the behavior of LLM-based agents which they refer to as LLM personas and simulate them to measure their personality traits.
Outcome:	The proposed model is based on the Big Five personality model and has been validated by human evaluations and automatic evaluations.

Copied to clipboard

Zheyuan Zhang, Lin Ge, Hongjiang Li, Weicheng Zhu, Chuxu Zhang, Yanfang Ye

An Evaluation Mechanism of LLM-based Agents on Manipulating APIs (2024.findings-emnlp)

Challenge:	Large language models (LLMs) have demonstrated remarkable capabilities across diverse tasks.
Approach:	They propose a framework that optimizes MAS prompts as a maximum a posteriori problem and then iteratively updates agent prompts.
Outcome:	The proposed framework surpasses manual and automated benchmarks in multiple tasks and provides general guidelines for building more reliable and principled multi-agent systems in the future.

Copied to clipboard

Bing Liu, Zhou Jianxiang, Dan Meng, Haonan Lu

Challenge:	Large Language Models (LLMs) have remarkable capabilities across a variety of tasks, such as language, mathematics, coding, and etc.
Approach:	They propose to decompose tool use capability into seven aspects and form a thorough evaluation schema for generic agents.
Outcome:	The proposed agent acts like a super-APP and can manipulate API-based tools.

Lying with Truths: Open-Channel Multi-Agent Collusion for Belief Manipulation via Generative Montage (2026.acl-long)

Copied to clipboard

Jinwei Hu, Xinmiao Huang, Youcheng Sun, Yi Dong, Xiaowei Huang

Hello Again! LLM-powered Personalized Agent for Long-term Dialogue (2025.naacl-long)

Challenge:	Large language models (LLMs) evolve to autonomous agents synthesizing real-time information, but their reasoning capabilities introduce an unexpected attack surface.
Approach:	They propose a framework that constructs deceptive narratives through adversarial debate and coordinated posting of evidence fragments, causing victims to internalize and propagate fabricated conclusions.
Outcome:	The proposed framework constructs deceptive narratives through adversarial debate and coordinated posting of evidence fragments, causing victims to internalize and propagate fabricated conclusions.

Copied to clipboard

Hao Li, Chenghao Yang, An Zhang, Yang Deng, Xiang Wang, Tat-Seng Chua

Natural Language Interfaces

Challenge:	Existing dialogue systems focus on brief single-session interactions, neglecting real-world needs for long-term companionship and personalized interactions.
Approach:	They propose a model-agnostic framework for long-term dialogue agents . they use event summary and persona management to enable reasoning .
Outcome:	The proposed framework incorporates three independently tunable modules dedicated to event perception, persona extraction, and response generation.

Agent-Pro: Learning to Evolve via Policy-Level Reflection and Optimization (2024.acl-long)

Copied to clipboard

Wenqi Zhang, Ke Tang, Hai Wu, Mengna Wang, Yongliang Shen, Guiyang Hou, Zeqi Tan, Peng Li, Yueting Zhuang, Weiming Lu

AgentCourt: Simulating Court with Adversarial Evolvable Lawyer Agents (2025.findings-acl)

Challenge:	Large Language Models (LLMs) are designed as specific task solvers with sophisticated prompt engineering, but are inherently incapacitating to address complex dynamic scenarios.
Approach:	They propose an LLM-based agent with policy-level reflection and optimization that can learn from interactive experiences and progressively elevate its behavioral policy.
Outcome:	The proposed agent outperforms vanilla LLM and specialized models in blackjack and Texas hold’em.

Copied to clipboard

Guhong Chen, Liyang Fan, Zihan Gong, Nan Xie, Zixuan Li, Ziqiang Liu, Chengming Li, Qiang Qu, Hamid Alinejad-Rokny, Shiwen Ni, Min Yang

MAQuA: Multi-outcome Adaptive Question-Asking for Mental Health using Item Response Theory (2026.eacl-long)

Challenge:	Existing legal language models struggle with dynamic courtroom interactions, resulting in overfitting to standardized legal tasks.
Approach:	They propose a new adversarial evolutionary approach for agents that performs dynamic knowledge learning and evolution through structured adversarials in a simulated courtroom program.
Outcome:	The proposed approach outperforms existing LLM-based models in three critical dimensions: cognitive agility, professional knowledge, and logical rigor.

Copied to clipboard

Vasudha Varadarajan, Hui Xu, Rebecca Astrid Böhme, Mariam Marlen Mirström, Sverker Sikström, H. Andrew Schwartz

MARK: Multi-agent Collaboration with Ranking Guidance for Text-attributed Graph Clustering (2025.findings-acl)

Challenge:	Evaluations of large language models (LLMs) indicate that such assessments are inconsistent and in many cases less accurate than dedicated condition-specific models with established psychometric validity.
Approach:	They propose a multi-outcome modeling and adaptive question-asking framework for simultaneous, multidimensional mental health screening that integrates language responses with item response theory and factor analysis.
Outcome:	Empirical results show that MAQuA reduces the number of assessment questions required for score stabilization by 50–87% compared to random ordering.

Copied to clipboard

Yiwei Fu, Yuxing Zhang, Chunchun Chen, JianwenMa JianwenMa, Quan Yuan, Rong-Cheng Tu, Xinli Huang, Wei Ye, Xiao Luo, Minghua Deng

EvoAgent: Towards Automatic Multi-Agent Generation via Evolutionary Algorithms (2025.naacl-long)

Challenge:	Existing approaches to cluster graphs with GNNs are limited due to label scarcity.
Approach:	They propose to leverage large language models to enhance text-attributed graph clustering by using three LLMs as ranking-based supervision signals.
Outcome:	The proposed approach generates reliable guidance using collaboration of three LLM-based agents as ranking-based supervision signals.

Copied to clipboard

Siyu Yuan, Kaitao Song, Jiangjie Chen, Xu Tan, Dongsheng Li, Deqing Yang

Time-aware ReAct Agent for Temporal Knowledge Graph Question Answering (2025.findings-naacl)

Challenge:	Existing work on extending specialized agents to multi-agent systems is dependent on human-designed frameworks, limiting the functional scope and scalability of agent systems.
Approach:	They propose a generic method to automatically extend specialized agents to multi-agent systems via evolutionary algorithm . they consider existing agent frameworks as the initial individual and apply evolutionary operators to generate multiple agents with diverse settings.
Outcome:	The proposed method can extend specialized agents to multi-agent systems . it can generate multiple agents with diverse settings, and improves performance across tasks .

Copied to clipboard

QianyiHu QianyiHu, Xinhui Tu, Guo Cong, Shunping Zhang

HEAL: Hybrid Enhancement with LLM-based Agents for Text-attributed Hypergraph Self-supervised Representation Learning (2025.findings-emnlp)

Challenge:	Existing solutions for temporal knowledge graph question answering lack sufficient temporal constraints in retrieval process.
Approach:	They propose a temporal knowledge graph question answering framework that integrates temporal constraints into information retrieval.
Outcome:	The proposed framework achieves a 41.3% improvement over the baseline model and a 32.2% gain compared to the Abstract Reasoning Induction (ARI) method.

Copied to clipboard

Ruochang Li, Xiao Luo, Zhiping Xiao, Wei Ju, Ming Zhang

EPO: Hierarchical LLM Agents with Environment Preference Optimization (2024.emnlp-main)

Challenge:	Existing approaches to enhance text-attributed hypergraph self-supervised learning are limited by label scarcity.
Approach:	They propose a data-centric approach that leverages large language models to enhance hypergraph self-supervised learning by integrating hyperedges into a self-representation framework.
Outcome:	The proposed approach generates informative nodes and hyperedges through multi-round interaction with LLM-based agents.

Copied to clipboard

Qi Zhao, Haotian Fu, Chen Sun, George Konidaris

Challenge:	Long-horizon decision-making tasks require extensive planning over multiple steps, maintaining coherence and goal orientation, which is difficult for LLMs that are typically designed for more immediate and localized predictions.
Approach:	They propose a hierarchical framework that decomposes complex tasks into manageable subgoals, utilizing separate LLMs for subgoal prediction and low-level action generation.
Outcome:	The proposed framework achieves first place on the ALFRED public leaderboard and demonstrates its potential to improve long-horizon decision-making in diverse environments.

AXIS: Efficient Human-Agent-Computer Interaction with API-First LLM-Based Agents (2025.acl-long)

Copied to clipboard

Junting Lu, Zhiyang Zhang, Fangkai Yang, Jue Zhang, Lu Wang, Chao Du, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang, Qi Zhang

Green & Sustainable NLP

Challenge:	Multimodal large language models (MLLMs) have enabled LLM-based agents to directly interact with application user interfaces (UIs), enhancing agents’ performance in complex tasks.
Approach:	They propose a novel agent framework that prioritizes actions through application programming interfaces over UI actions and facilitates the creation and expansion of APIs through automated exploration of applications.
Outcome:	The proposed framework reduces task completion time by 65%-70% and cognitive workload by 38%-53% while maintaining accuracy of 97%-98% compared to humans.

CoMet: Metaphor-Driven Covert Communication for Multi-Agent Language Games (2025.acl-long)

Copied to clipboard

Shuhang Xu, Fangwei Zhong

Explainability & Interpretability in NLP

Challenge:	Metaphors are crucial for humans to express complex or subtle ideas by comparing one concept to another, often from a different domain.
Approach:	They propose a framework that enables LLMs to engage in metaphor processing by combining hypothesis-based metaphor reasoner and metaphor generator.
Outcome:	The proposed framework enhances agents' ability to interpret and apply metaphors in language games.

Law in Silico: Simulating Legal Society with LLM-Based Agents (2026.findings-acl)

Copied to clipboard

Yiding Wang, Yuxuan Chen, Fanxu Meng, Xifan Chen, Xiaolei Yang, Muhan Zhang

Challenge:	Large Language Models (LLMs) are powerful tools for legal simulation, but their application remains underexplored.
Approach:	They propose a unified LLM-based agent framework for simulating legal scenarios . they calibrate agent behaviors against real-world crime data .
Outcome:	The proposed framework calibrates agent behaviors against real-world crime data.

Hypothesis Generation for Materials Discovery and Design Using Goal-Driven and Constraint-Guided LLM Agents (2025.findings-naacl)

Copied to clipboard

Shrinidhi Kumbhar, Venkatesh Mishra, Kevin Coutinho, Divij Handa, Ashif Iquebal, Chitta Baral

Tooling or Not Tooling? The Impact of Tools on Language Agents for Chemistry Problem Solving (2025.findings-naacl)

Challenge:	Recent research has leveraged Large Language Models to accelerate materials discovery and design.
Approach:	They propose a dataset that features goals, constraints, and methods for designing real-world applications and a method that emulates the process a materials scientist would use to evaluate a hypothesis critically.
Outcome:	The proposed method emulates the process a materials scientist would use to evaluate a hypothesis critically.

Copied to clipboard

Botao Yu, Frazier N. Baker, Ziru Chen, Garrett Herb, Boyu Gou, Daniel Adu-Ampratwum, Xia Ning, Huan Sun

Challenge:	Existing evaluations of large language models (LLMs) with tools are limited and qualitative . existing evaluations have been limited and only focus on 14 tasks focusing on compound synthesis.
Approach:	They propose to develop an enhanced chemistry agent over ChemCrow to improve chemistry problem solving by integrating tools into LLMs.
Outcome:	The proposed agent does not consistently outperform its base LLMs without tools on specialized chemistry tasks and general chemistry questions.

Reasoning as Gradient: Scaling MLE Agents Beyond Tree Search (2026.findings-acl)

Copied to clipboard

Yifei Zhang, Xu Yang, Xiao Yang, Bowen Xian, Qizheng Li, Shikai Fang, Jingyuan Li, Jian Wang, Minrui Xu, Yuge Zhang, Weiqing Liu, Jiang Bian

Ready Jurist One: Benchmarking Language Agents for Legal Intelligence in Dynamic Environments (2026.acl-long)

Challenge:	LLM-based agents for machine learning engineering rely on tree search to rank candidates.
Approach:	They propose an LLM-based agent that operationalizes gradient-based optimization.
Outcome:	The proposed agent achieves a state-of-the-art 35.1% any-medal rate on MLE-Bench with a limited budget on a single GPU.

Copied to clipboard

Zheng Jia, Shengbin Yue, Wei Chen, Siyuan Wang, Yidong Liu, Zejun Li, Yun Song, Zhongyu Wei

Challenge:	Existing benchmarks for legal intelligence are limited to static evaluation paradigms or simplified scenarios.
Approach:	They introduce J1-ENVS, the first interactive and dynamic legal environment tailored for LLM-based agents.
Outcome:	The proposed framework assesses task performance and procedural compliance across legal proficiency levels.

Mobile-Bench: An Evaluation Benchmark for LLM-based Mobile Agents (2024.acl-long)

Copied to clipboard

Shihan Deng, Weikai Xu, Hongda Sun, Wei Liu, Tao Tan, Liujianfeng Liujianfeng, Ang Li, Jian Luan, Bin Wang, Rui Yan, Shuo Shang

Investigating and Extending Homans’ Social Exchange Theory with Large Language Model based Agents (2025.acl-long)

Challenge:	Existing benchmarks for LLM-based mobile agents are insufficient to evaluate their capabilities.
Approach:	They propose a benchmark to evaluate LLM-based mobile agents' planning capabilities . they expand UI operations by incorporating 103 APIs to accelerate task completion .
Outcome:	The proposed benchmarks are based on 103 collected APIs and real user queries . the data is categorized into three distinct groups: SAST, SAMT, and MAMT .

Copied to clipboard

Lei Wang, Zheqing Zhang, Xu Chen

Embracing Imperfection: Simulating Students with Diverse Cognitive Levels Using LLM-based Agents (2025.acl-long)

Challenge:	Social exchange theory (SET) is widely recognized as a basic framework for understanding human interactions and interactions.
Approach:	They propose to use large language models to study Homans’ social exchange theory (SET) by constructing a virtual society composed of three LLM agents and having them engage in a social exchange game to observe their behaviors.
Outcome:	The proposed model extends Homans’ SET with LLM-based agents and demonstrates consistency between the agent and human behavior.

Copied to clipboard

Tao Wu, Jingyuan Chen, Wang Lin, Mengze Li, Yumeng Zhu, Ang Li, Kun Kuang, Fei Wu

Challenge:	Large language models (LLMs) are becoming increasingly popular in education, enabling researchers to simulate students' learning patterns and learning patterns.
Approach:	They propose a training-free framework for student simulation that takes into account student cognitive diversity and realism.
Outcome:	The proposed model outperforms baseline models and achieves 100% improvement in simulation accuracy and realism.

MMedAgent: Learning to Use Medical Tools with Multi-modal Agent (2024.findings-emnlp)

Copied to clipboard

Binxu Li, Tiankai Yan, Yuanting Pan, Jie Luo, Ruiyang Ji, Jiayuan Ding, Zhe Xu, Shilong Liu, Haoyu Dong, Zihao Lin, Yixin Wang

WebCoderBench: Benchmarking Web Application Generation with Comprehensive and Interpretable Evaluation Metrics (2026.acl-long)

Challenge:	Multi-modal Large Language Models (MLLMs) exhibit limited generality and often fall short when compared to specialized models.
Approach:	They propose a multi-modal medical agent that picks the most suitable medical tools based on user inputs.
Outcome:	The proposed agent performs better than open-source models and the closed-source model, GPT-4o.

Copied to clipboard

Chenxu Liu, Yingjie Fu, Wei Yang, Ying Zhang, Tao Xie

Explainability & Interpretability in NLP

Challenge:	Web applications (web apps) are a key arena for large language models to demonstrate their code generation capabilities and commercial potential.
Approach:	a new benchmark for large language models (LLMs) is designed to provide real-world user requirements and generalizable evaluation metrics.
Outcome:	a new benchmark for large language models (LLMs) provides a real-world, generalizable, and interpretable evaluation score . the benchmark measures user requirements, expression styles and human-preference-aligned weights . a web application can be used to demonstrate its commercial potential, authors say .

A Survey of LLM-based Agents in Medicine: How far are we from Baymax? (2025.findings-acl)

Copied to clipboard

Wenxuan Wang, Zizhan Ma, Zheng Wang, Chenghan Wu, Jiaming Ji, Wenting Chen, Xiang Li, Yixuan Yuan

Large Language Models for IT Automation Tasks: Are We There Yet? (2026.findings-acl)

Challenge:	Large Language Models (LLMs) are transforming healthcare through their ability to understand and assist with medical tasks.
Approach:	They analyze system profiles, clinical planning, medical reasoning frameworks, and external capacity enhancement.
Outcome:	The findings highlight the future directions in medical reasoning, physical system integration, and training simulations.

Copied to clipboard

Md. Mahadi Hassan, John Salvador, Akond Ashfaque Ur Rahman, Santu Karmaker

AgentMark: Utility-Preserving Behavioral Watermarking for Agents (2026.acl-long)

Challenge:	Existing benchmarks rely on synthetic tasks that fail to capture the needs of practitioners who use IT automation tools.
Approach:	They evaluate 14 open-source and 3 proprietary LLMs and find that GPT-4.1-Mini achieves the best pass@10 rate of 23.9%, while Claude-3.5-Sonnet achieves best pass @1 performance.
Outcome:	The evaluated LLMs perform poorly in 126 tasks and show that they lack state reconciliation capabilities and lack module knowledge.

Copied to clipboard

Kaibo Huang, Jin Tan, Yukun Wei, Wanling Li, Zipei Zhang, Hui Tian, Zhongliang Yang, Linna Zhou

Challenge:	Recent advances in large language models (LLMs) have improved text generation and reasoning.
Approach:	They propose a behavioral watermarking framework that embeds multi-bit identifiers into planning decisions while preserving utility.
Outcome:	The proposed framework embeds multi-bit provenance into planning decisions while preserving utility.

From Heard to Lived Opinions: Simulating Opinion Dynamics with Grounded LLM Agents in Economic Environments (2026.findings-acl)

Copied to clipboard

Ryuji Hashimoto, Masahiro Kaneko, Ryosuke Takata, Takehiro Takayanagi, Kiyoshi Izumi

Sentiment Analysis

Challenge:	Existing studies on opinion dynamics (OD) focus primarily on opinion exchange, with opinion change driven by linguistic interaction.
Approach:	They propose a OD simulation framework that grounds LLM-based agents in an economic environment and allows them to act and receive environmental feedback.
Outcome:	The proposed framework shows that LLM-based agents can act and receive environmental feedback at both individual and population levels while generating larger distributional shifts.

TrustAgent: Towards Safe and Trustworthy LLM-based Agents (2024.findings-emnlp)

Copied to clipboard

Wenyue Hua, Xianjun Yang, Mingyu Jin, Zelong Li, Wei Cheng, Ruixiang Tang, Yongfeng Zhang

AI-LieDar : Examine the Trade-off Between Utility and Truthfulness in LLM Agents (2025.naacl-long)

Challenge:	Existing LLMs are primarily used for simple text-related tasks, but LLM-based agents can undertake more complex tasks that require planning and interaction with the physical world and humans.
Approach:	They propose an Agent-Constitution-based agent framework with a particular focus on improving the LLM-based agents' safety.
Outcome:	The proposed framework can enhance an LLM agent’s safety across multiple domains by identifying and mitigating potential dangers during the planning process.

Copied to clipboard

Zhe Su, Xuhui Zhou, Sanketh Rangreji, Anubha Kabra, Julia Mendelsohn, Faeze Brahman, Maarten Sap

Challenge:	LieDar is a framework to study how LLM-based agents navigate these scenarios in a multi-turn interactive setting.
Approach:	They propose a framework to study how LLM-based agents navigate these scenarios in an interactive multi-turn setting.
Outcome:	The proposed framework shows that all models are truthful less than 50% of the time, although truthfulness and goal achievement rates vary across models.

SDPO: Segment-Level Direct Preference Optimization for Social Agents (2025.acl-long)

Copied to clipboard

Aobo Kong, Wentao Ma, Shiwan Zhao, Yongbin Li, Yuchuan Wu, Ke Wang, Xiaoqian Liu, Qicheng Li, Yong Qin, Fei Huang

Challenge:	Direct Preference Optimization (DPO) has proven effective in aligning LLM behavior with human preferences across various tasks, but is limited in multi-turn social interactions.
Approach:	They propose a method which dynamically selects key segments within interactions to optimize multi-turn agent behavior.
Outcome:	The proposed methods outperform existing methods and proprietary LLMs on the SOTOPIA benchmark and show that they can improve social intelligence.

Faithful, Unfaithful or Ambiguous? Multi-Agent Debate with Initial Stance for Summary Evaluation (2025.naacl-long)

Copied to clipboard

Mahnaz Koupaee, Jake W. Vincent, Saab Mansour, Igor Shalyminov, Han He, Hwanjun Song, Raphael Shu, Jianfeng He, Yi Nian, Amy Wing-mei Wong, Kyu J. Han, Hang Su

Challenge:	Existing approaches to evaluate faithfulness of summaries are often fooled by the fluency of the text and struggle with identifying errors.
Approach:	They propose an approach to summary faithfulness evaluation where multiple LLM-based agents are assigned initial stances and forced to come up with a reason to justify belief.
Outcome:	The proposed approach can identify ambiguities and have even stronger performance on non-ambiguous summaries.

Agentic Knowledgeable Self-awareness (2025.acl-long)

Copied to clipboard

Shuofei Qiao, Zhisong Qiu, Baochang Ren, Xiaobin Wang, Xiangyuan Ru, Ningyu Zhang, Xiang Chen, Yong Jiang, Pengjun Xie, Fei Huang, Huajun Chen

Challenge:	Large Language Models (LLMs) have achieved considerable performance across various agentic planning tasks.
Approach:	They propose a data-centric approach that applies agents with knowledgeable self-awareness like humans to a heuristic situation judgement criterion to mark special tokens on their self-explored trajectories for collecting training data.
Outcome:	The proposed paradigm outperforms baseline models on various tasks with minimal external knowledge.

FlowBench: Revisiting and Benchmarking Workflow-Guided Planning for LLM-based Agents (2024.findings-emnlp)

Copied to clipboard

Ruixuan Xiao, Wentao Ma, Ke Wang, Yuchuan Wu, Junbo Zhao, Haobo Wang, Fei Huang, Yongbin Li

Challenge:	LLM-based agents are susceptible to undesired planning hallucinations when lacking specific knowledge for expertise-intensive tasks.
Approach:	They propose a benchmark to evaluate the efficacy of workflow-guided agent planning by formalizing different formats of workflow knowledge.
Outcome:	The proposed benchmark aims to improve the planning reliability of LLM-based agents by incorporating external workflow-related knowledge.

Coarse-to-Fine Grounded Memory for LLM Agent Planning (2025.emnlp-main)

Copied to clipboard

Wei Yang, Jinwei Xiao, Hongming Zhang, Qingyang Zhang, Yanna Wang, Bo Xu

Challenge:	Existing methods to enhance LLM with offline experiences or online trajectory analysis focus on single-granularity memory derived from dynamic environmental interactions.
Approach:	They propose a framework that grounds coarse-to-fine memories with LLM to enable flexible adaptation to diverse scenarios.
Outcome:	Extensive experiments on AlfWorld, Webshop and ScienceWorld show that the proposed framework outperforms baselines and comprehensively optimizes memory-enhanced LLM Agent system.

Controllable Memory Usage: Balancing Anchoring and Innovation in Long-Term Human–Agent Interaction (2026.acl-long)

Copied to clipboard

Zisu Huang, Muzhao Tian, Xiaohua Wang, Jingwen Xu, Zhengkang Guo, Qi Qian, Kaitao Song, Jiakang Yuan, Changze Lv, Xiaoqing Zheng

Challenge:	Existing systems that use memory as an "all-or-nothing" approach to memory usage are often static and rely on experience-following tendencies.
Approach:	They propose a framework that allows users to dynamically regulate memory reliance by adding context into the model's prompt.
Outcome:	The proposed model outperforms prompting and memory masking strategies in multiple scenarios.

Spatial-Agent: Agentic Geo-spatial Reasoning with Scientific Core Concepts (2026.acl-long)

Copied to clipboard

Riyang Bao, Cheng Yang, Dazhou Yu, Zhexiang Tang, Gengchen Mai, Liang Zhao

ControlMath: Controllable Data Generation Promotes Math Generalist Models (2024.emnlp-main)

Challenge:	Existing LLM-based agents lack inherent spatial awareness, relying on web search or text matching while hallucinating spatial relationships.
Approach:	They propose a spatial-based agent that can perform real-world geospatial computations . they use natural-language questions to parse into executable workflows based on geoFlow Graphs - directed acyclic graphs with nodes corresponding to spatial concepts and edges representing transformations.
Outcome:	The proposed agent outperforms existing baselines on MapEval-API and MapQA benchmarks while producing interpretable and executable geospatial workflows.

Copied to clipboard

Nuo Chen, Ning Wu, Jianhui Chang, Linjun Shou, Jia Li

SAMem: State-Aware Memory as a Fine-Grained Memory for LLM Agents in Decision-Making (2026.findings-acl)

Challenge:	Currently, mathematical reasoning is one of the most challenging areas for closed-source LLMs.
Approach:	They propose an iterative method involving an equation-generator module and two LLM-based agents that generate diverse equations and transform them into math word problems.
Outcome:	The proposed method enables the generation of diverse math problems, not limited to specific domains or distributions.

Copied to clipboard

Tong Wang, Pei Xu, Shiyue Cao, Likun Yang, Daipeng Li, Jianbin Jiao, Kaiqi Huang

Invocation Refiner: A Plug-and-Play Module for Rectifying LLM Tool Invocations (2026.findings-acl)

Challenge:	Existing experiential memory approaches rely on task-level memory, but this lacks the situational alignment required for complex multi-step decision-making.
Approach:	They propose a new fine-grained memory paradigm that aligns memory retrieval with the current state instead of storing and reusing globally shared experiences.
Outcome:	Experiments on complex decision-making benchmarks show that the proposed state-aware memory outperforms existing experiential memory approaches and significantly improves task-solving efficiency.

Copied to clipboard

Qirui Jiao, Dian Jiao, Nan Du, Ying Shen, Liang Lin

SeMob: Semantic Synthesis for Dynamic Urban Mobility Prediction (2025.emnlp-main)

Challenge:	Large Language Models (LLMs) have shown remarkable capabilities in Tool-Integrated Reasoning (TIR) however, the practical application is often hindered by frequent errors in tool invocations, such as incorrect tool names, invalid parameters, wrong tool-call order, or malformed invocation formats.
Approach:	They propose a specialized post-processing module that performs independent reasoning on the input of a frozen upstream LLM and an advanced RL algorithm to improve the tool-use reliability of base LLMs.
Outcome:	The proposed module improves task completion rates and invocation accuracy over the raw outputs of various upstream LLMs on a diverse set of tool-use and reasoning benchmarks.

Copied to clipboard

Runfei Chen, Shuyang Jiang, Wei Huang

Challenge:	Existing spatiotemporal models struggle to interpret and adapt to abrupt changes caused by external events.
Approach:	They propose a LLM-powered semantic synthesis pipeline that extracts spatiotemporally related text from online texts and integrates it with spatio-temporal data.
Outcome:	The proposed framework achieves maximal reductions of 13.92% in MAE and 11.12% in RMSE compared to the spatiotemporal model.

ToolCPT: Improving Tool Utilization in LLM Agents via Continuous Pre-training (2026.findings-acl)

Copied to clipboard

Yifan Yang, Jinghui Lu, null Evadeng, Ao Yang, Peijie Yu, TingHao YU, Feng Zhang

Tree-Notebook: A Context-Aware Agent with Tree Search and Entropy-Aware Data Shadow for Interactive Data Science (2026.findings-acl)

Challenge:	Current approaches to enhancing tool use for LLM-based agents focus on post-training fine-tuning or test-time context extension.
Approach:	They propose to enhance tool knowledge for LLM-based agents during continuous pre-training . they curate 5.1 million code artifacts from large-scale, high-quality code repositories .
Outcome:	The proposed model outperforms existing methods on out-of-distribution tools on multiple benchmarks.

Copied to clipboard

Junkun Qiu, Min Huang, Qinghai Miao

Green & Sustainable NLP

Challenge:	Experimental results show that Tree-Notebook achieves state-of-the-art (SOTA) performance on InfiAgent-DABench and DSBench.
Approach:	They propose an agentic framework that mimics the iterative cognitive process of human data scientists.
Outcome:	The proposed framework achieves state-of-the-art (SOTA) performance on InfiAgent-DABench and DSBench.

From Capabilities to Performance: Evaluating Key Functional Properties of LLM Architectures in Penetration Testing (2025.emnlp-main)

Copied to clipboard

Lanxiao Huang, Daksh Dave, Tyler Cody, Peter A. Beling, Ming Jin

Beyond Demographics: Aligning Role-playing LLM-based Agents Using Human Belief Networks (2024.findings-emnlp)

Challenge:	Large Language Models (LLMs) have been explored for automating or enhancing penetration testing tasks, but their effectiveness and reliability remain open questions.
Approach:	They evaluate multiple LLM-based agents across realistic penetration testing scenarios . they also examine impact of core functional capabilities on agent success .
Outcome:	The proposed models improve agent performance in multi-step and real-time penetration testing scenarios.

Copied to clipboard

Yun-Shiuan Chuang, Krirk Nirunwiroj, Zach Studdiford, Agam Goyal, Vincent Frigo, Sijia Yang, Dhavan Shah, Junjie Hu, Timothy Rogers

Challenge:	Existing large language models can be prompted to role-play as individuals with particular demographic traits, but results are often human-like.
Approach:	They found that seeding LLM-based agents with a single belief improved alignment . they say that role-playing based on demographic information does not improve alignment a .
Outcome:	The proposed approach improves LLM alignment with human behavior . seeding agents with a single belief improves alignment for topics related to the belief network .

LLMs for Bayesian Optimization in Scientific Domains: Are We There Yet? (2025.findings-emnlp)

Copied to clipboard

Rushil Gupta, Jason Hartford, Bang Liu

Challenge:	Large language models have been proposed as general-purpose agents for experimental design . eval: LLMs show no sensitivity to experimental feedback.
Approach:	They propose a method that combines LLM prior knowledge with nearest-neighbor sampling to guide the design of experiments.
Outcome:	The proposed method outperforms classical methods in the design of experiments.

Enhancing Open-Domain Task-Solving Capability of LLMs via Autonomous Tool Integration from GitHub (2025.acl-long)

Copied to clipboard

Bohan Lyu, Xin Cong, Heyang Yu, Pan Yang, Cheng Qian, Zihe Wang, Yujia Qin, Yining Ye, Yaxi Lu, Chen Qian, Zhong Zhang, Yukun Yan, Yankai Lin, Zhiyuan Liu, Maosong Sun

Dynamics of Cognitive Heterogeneity: Investigating Behavioral Biases in Multi-Stage Supply Chains with LLM-Based Simulation (2026.acl-long)

Challenge:	Existing approaches lack flexibility to address diverse and ever-evolving user queries in open domains.
Approach:	They propose to evaluate LLMs on open-domain knowledge that requires tools to solve diverse and ever-evolving user queries.
Outcome:	The proposed system outperforms baselines in the open domain task-solving benchmark.

Copied to clipboard

Jiuyun Jiang, Yuecheng Hong, Bo Yang, Jin Yang, Guangxin Jiang, Xiaomeng Guo, Guang Xiao

PersonaLens: A Benchmark for Personalization Evaluation in Conversational AI Assistants (2025.findings-acl)

Challenge:	Recent advances in Large Language Models (LLMs) allow repeatable experiments in which individual characteristics can be precisely defined.
Approach:	They propose a scalable experimental paradigm using Large Language Models to simulate multi-stage supply chain dynamics.
Outcome:	The proposed model systematically replicates and validates the results of a behavioral simulation on agents in multi-stage supply chain dynamics.

Copied to clipboard

Zheng Zhao, Clara Vania, Subhradeep Kayal, Naila Khan, Shay B Cohen, Emine Yilmaz

Dialogue Systems & Conversational Agents

Challenge:	Existing personalization benchmarks focus on chit-chat, non-conversational tasks, or narrow domains, failing to capture complexities of personalized task-oriented assistance.
Approach:	They propose a benchmark to evaluate personalization in task-oriented AI assistants . the benchmark features user profiles equipped with rich preferences and interaction histories .
Outcome:	The proposed benchmark features user profiles equipped with rich preferences and interaction histories . it also features a judge agent and user agent that employs the LLM-as-a-Judge paradigm .

AgentRM: Enhancing Agent Generalization with Reward Modeling (2025.acl-long)

Copied to clipboard

Yu Xia, Jingru Fan, Weize Chen, Siyu Yan, Xin Cong, Zhong Zhang, Yaxi Lu, Yankai Lin, Zhiyuan Liu, Maosong Sun

CAIR: Counterfactual-based Agent Influence Ranker for Agentic AI Workflows (2025.emnlp-main)

Challenge:	Existing LLM-based agents have strong performance on held-in tasks, but their generalizability to unseen tasks remains poor.
Approach:	They propose a reward-based generalizable reward model to guide the policy model for effective test-time search.
Outcome:	The proposed agentRM outperforms existing agents on held-in tasks by 8.8 points on average.

Copied to clipboard

Amit Giloni, Chiara Picardi, Roy Betser, Shamik Bose, Aishvariya Priya Rathina Sabapathy, Roman Vainshtein

Challenge:	Existing methods to assess the influence of each agent on the AAW’s output perform only static structural analysis, which is unsuitable for inference time execution.
Approach:	They propose to use an LLM-based agent influence Ranker to assess the influence level of each agent on the AAW's output and determine which agents are the most influential.
Outcome:	The proposed method outperforms baseline methods and produces consistent rankings and relevancy of downstream tasks.

MemBench: Towards More Comprehensive Evaluation on the Memory of LLM-based Agents (2025.findings-acl)

Copied to clipboard

Haoran Tan, Zeyu Zhang, Chen Ma, Xu Chen, Quanyu Dai, Zhenhua Dong

Challenge:	Recent studies have highlighted the significance of memory mechanisms in LLM-based agents, which enable them to store observed information and adapt to dynamic environments.
Approach:	They propose a dataset and benchmark to evaluate the memory capability of LLM-based agents from multiple aspects including their effectiveness, efficiency, and capacity.
Outcome:	The proposed benchmark incorporates factual memory and reflective memory as different levels, and proposes participation and observation as various interactive scenarios.

Data Interpreter: An LLM Agent for Data Science (2025.findings-acl)

Copied to clipboard

Sirui Hong, Yizhang Lin, Bang Liu, Bangbang Liu, Binhao Wu, Ceyao Zhang, Danyang Li, Jiaqi Chen, Jiayi Zhang, Jinlin Wang, Li Zhang, Lingyao Zhang, Min Yang, Mingchen Zhuge, Taicheng Guo, Tuo Zhou, Wei Tao, Robert Tang, Xiangtao Lu, Xiawu Zheng, Xinbing Liang, Yaying Fei, Yuheng Cheng, Yongxin Ni, Zhibin Gou, Zongze Xu, Yuyu Luo, Chenglin Wu

From Prediction to Intervention: Personalized Meal-Level Glucose Regulation via an LLM Agent (2026.findings-acl)

Challenge:	Large Language Models (LLMs) excel in various domains but face challenges when applied to data science workflows due to their complex, multi-stage nature.
Approach:	They propose a hierarchical graph-based agent that represents complexity and a progressive strategy for step-by-step verification, refinement, and consistent context management.
Outcome:	The proposed agent surpasses state-of-the-art baselines on the MATH dataset and performs better on InfiAgent-DABench.

Copied to clipboard

Mingyu Huang, Weiqing Min, Ying Jin, Yilin Wang, Shuqiang Jiang

Challenge:	Existing approaches to individualized glucose regulation are generic and do not account for individual-specific glucose dynamics.
Approach:	They propose a physio-feedback agentic loop that integrates individualized absorption modeling with dietary intervention to regulate glucose response.
Outcome:	The proposed system improves prediction accuracy and reduces glucose excursions.

Metagent-P: A Neuro-Symbolic Planning Agent with Metacognition for Open Worlds (2025.findings-acl)

Copied to clipboard

YanfangZhou YanfangZhou, Yuntao Liu, Xiaodong Li, Yongqiang Zhao, Xintong Wang, Jinlong Tian, Zhenyu Li, Xinhai Xu

Cognitive Scaffold: From Fluid Context to Crystallized Memory for Long-Horizon DeepResearch Agents (2026.acl-long)

Challenge:	Recent advances in large language models (LLMs) show promising potential through their world knowledge and language processing capabilities in open-world planning.
Approach:	They propose a framework that integrates the world knowledge of large language models, symbolic reasoning capabilities of cognitive architectures, and metacognition to improve experience utilization.
Outcome:	The proposed framework outperforms current state-of-the-art methods in Minecraft and reduces the average replanning counts by 34% and exceeds the human success rate by 18.96%.

Copied to clipboard

Qiuyuan Ai, Zenghuang Fu, Zhaoyang Li, Ping Jiang, Haoyu Wu, Jie Song, Guannan He

Stop Fixating on Prompts: Reasoning Hijacking and Constraint Tightening for Red-Teaming LLM Agents (2026.acl-long)

Challenge:	Scaling LLM-based agents to long-horizon deep research is constrained by context-noise trade-off . solving a single query may require hundreds of interactions with noisy environments .
Approach:	They propose a factorized memory architecture that decouples the cognitive state into a Fluid Working Context for immediate reasoning and a persistent Knowledge Graph for long-term retention.
Outcome:	The Cognitive Scaffold outperforms baselines on Xbench-DeepSearch, BrowseComp-ZH, and GAIA . it achieves 74.7% Avg@3 and 87.0% Pass@3 on xbench, browseComp, and 88.3% Pass@3.

Copied to clipboard

Yanxu Mao, Peipei Liu, Tiehan Cui, Congying Liu, Mingzhe Xing, Datao You

Low-Resource NLP

Challenge:	Existing red-team methods rely on modifying user prompts, which lack adaptability to new data and may impact the agent’s performance.
Approach:	They propose a framework that implicitly manipulates the agent’s reasoning trajectory and memory retrieval with three key stages: Trigger Extraction, Reasoning Hijacking, and Constraint Tightening.
Outcome:	The proposed framework shows outstanding performance in cross-model and cross-scenario environments.

Is this the real life? Is this just fantasy? The Misleading Success of Simulating Social Interactions With LLMs (2024.emnlp-main)

Copied to clipboard

Xuhui Zhou, Zhe Su, Tiwalayo Eisape, Hyunwoo Kim, Maarten Sap

Challenge:	Recent advances in large language models have enabled richer social simulations . however, the role of information asymmetry in these simulations has been overlooked .
Approach:	They develop an evaluation framework to simulate social interactions with LLMs in different settings.
Outcome:	The proposed framework performs better in unrealistic, omniscient simulation settings but struggles in those with information asymmetry.

Multiple LLM Agents Debate for Equitable Cultural Alignment (2025.acl-long)

Copied to clipboard

Dayeon Ki, Rachel Rudinger, Tianyi Zhou, Marine Carpuat

Challenge:	Recent efforts focus on single-LLM, single-turn generation approaches, but it can be challenging for any single model to support all cultures equally well.
Approach:	They propose to exploit the complementary strengths of multiple LLMs to promote cultural adaptability.
Outcome:	The proposed model improves accuracy and cultural group parity over single-LLM models.

Enabling Agents to Communicate Entirely in Latent Space (2026.acl-long)

Copied to clipboard

Zhuoyun Du, Runze Wang, Huiyu Bai, Zouying Cao, Xiaoyong Zhu, Yu Cheng, Bo Zheng, Wei Chen, Haochao Ying

Challenge:	Natural language is the de facto communication medium for LLM-based agents, but it presents a fundamental constraint . natural language downsampling limits the depth and nuance of information that can be transmitted . et al.: inter-agent latent space communication is a promising paradigm for solving complex tasks .
Approach:	They propose a paradigm that leverages the last hidden states of an LLM as a representation of its thought for direct communication.
Outcome:	The proposed paradigm outperforms fine-tuned chain-of-thought prompting and single-agent baselines even across heterogeneous models.

PACE: Predictive Adaptive Context Extraction for Long-Horizon LLM Agents (2026.acl-long)

Copied to clipboard

Lei Wei, Xiao Peng, null Tt, Guannan Zhang, Chenhao Jiang, Hongyu Li, Lanbo Lin, Yuanwu Xu, Jiayao Liu, Kesu Wang, Bin Wang

Information Extraction & Text Mining

Challenge:	Large Language Model (LLM) agents struggle with ultra-long-horizon tasks requiring hundreds or thousands of interaction steps.
Approach:	They propose a framework that reconceptualizes context management as a Next Step Prediction problem.
Outcome:	The proposed framework improves task success rates and robust cross-lingual performance.

Runaway is Ashamed, But Helpful: On the Early-Exit Behavior of Large Language Model-based Agents in Embodied Environments (2025.findings-emnlp)

Copied to clipboard

Qingyu Lu, Liang Ding, Siyi Cao, Xuebo Liu, Kanjian Zhang, Jinxia Zhang, Dacheng Tao

A Survey on Evaluation of LLM-based Agents (2026.findings-acl)

Challenge:	Experiments with 4 different LLMs across 5 embodied environments show significant efficiency improvements, with only minor drops in agent performance.
Approach:	They propose an intrinsic method that injects exit instructions during generation and an extransic system that verifies task completion to determine when to halt an agent’s trial.
Outcome:	The proposed method injects exit instructions during generation and an exit method verifies task completion to determine when to halt an agent’s trial.

Copied to clipboard

Asaf Yehudai, Lilach Eden, Alan Li, Guy Uziel, Yilun Zhao, Roy Bar-Haim, Arman Cohan, Michal Shmueli-Scheuer

Challenge:	This paper provides the first comprehensive survey of evaluation methods for LLM-based agents . LLMs are static, having fixed knowledge, and confined to text-to-text interaction.
Approach:	They analyze the evaluation of LLM-based agents across five perspectives . they identify current trends and key gaps in evaluation methods .
Outcome:	The proposed evaluation frameworks and tools are based on five perspectives . the results highlight current trends and identify gaps in future research .

StructuThink: Reasoning with Task Transition Knowledge for Autonomous LLM-Based Agents (2025.findings-emnlp)

Copied to clipboard

Haiyu Zhao, Zhenyu Guo, Chunhong Zhang, Ziyu Zhou, Zheng Hu

OptiSeq: Ordering Examples On-The-Fly for In-Context Learning (2025.findings-emnlp)

Challenge:	StructuThink framework enhances LLMs' ability to ground decisions in domain-specific scenarios.
Approach:	They propose a knowledge-structured reasoning framework that enhances LLM-based agents with explicit decision constraints.
Outcome:	The proposed framework achieves higher task success rates and more efficient action sequences than baseline methods.

Copied to clipboard

Rahul Atul Bhope, Praveen Venkateswaran, K. R. Jayaram, Vatche Isahagian, Vinod Muthusamy, Nalini Venkatasubramanian

SILO-BENCH: A Scalable Environment for Evaluating Distributed Coordination in Multi-Agent LLM Systems (2026.acl-long)

Challenge:	In-context-learning (ICL) is fragile and requires a lot of examples to perform.
Approach:	They propose a purely inference-time, dataset-free optimization method that efficiently determines the best example order.
Outcome:	The proposed method improves in-context-learning accuracy by 5.5 - 10.5 percentage points across multiple tasks.

Copied to clipboard

Yuzhe Zhang, Feiran Liu, Yi Shan, Xinyi Huang, Xin Yang, Yueqi Zhu, Xuxin Cheng, Cao Liu, Ke Zeng, Terry Jingchen Zhang, Wenyuan Jiang

Challenge:	Existing benchmarks conflate coordination ability with role-based priors.
Approach:	They propose a role-free benchmark for evaluating free-form collaboration under information silos.
Outcome:	The proposed benchmark systematically probes coordination capabilities under information silos using 54 configurations and 3 frontier LLMs.

AgentGym: Evaluating and Training Large Language Model-based Agents across Diverse Environments (2025.acl-long)

Copied to clipboard

Zhiheng Xi, Yiwen Ding, Wenxiang Chen, Boyang Hong, Honglin Guo, Junzhe Wang, Xin Guo, Dingwen Yang, Chenyang Liao, Wei He, Songyang Gao, Lu Chen, Rui Zheng, Yicheng Zou, Tao Gui, Qi Zhang, Xipeng Qiu, Xuanjing Huang, Zuxuan Wu, Yu-Gang Jiang

DEFT: Demystifying VLN Failures via a Unified Dual-View Explainability Framework for LLM-based Agents (2026.acl-long)

Challenge:	Large language models (LLMs) are promising foundations to build generally-capable agents . however, the community lacks a unified interactive framework that covers diverse environments for comprehensive evaluation of agents.
Approach:	They propose a framework that features 7 real-world scenarios, 14 environments, and 89 tasks for unified, real-time, and concurrent agent interaction.
Outcome:	The proposed framework features 7 real-world scenarios, 14 environments, and 89 tasks for unified, real-time, and concurrent agent interaction.

Copied to clipboard

Yawen Wang, Yihan Dai, Jianming Chen, Junjie Wang, Qing Wang

Explainability & Interpretability in NLP

Challenge:	Existing interpretability methods isolate temporal criticality from feature salience, creating an alignment gap and failing to account for the behavioral instability of black-box agents.
Approach:	They propose a unified dual-view framework that jointly analyzes when a decision is pivotal and what visual evidence grounds it.
Outcome:	Extensive experiments on MatterPort3D show that DEFT outperforms baselines in both temporal and feature fidelity.

SafeAgent: Safeguarding LLM Agents via an Automated Risk Simulator (2026.acl-long)

Copied to clipboard

Xueyang Zhou, Weidong Wang, Lin Lu, Jiawen Shi, Guiyao Tie, Xu Yongtian, Lixing Chen, Pan Zhou, Neil Zhenqiang Gong, Lichao Sun

Challenge:	SafeAgent improves agent safety through fully automated synthetic data generation.
Approach:	They propose a framework that improves agent safety through fully automated synthetic data generation.
Outcome:	The proposed framework outperforms closed-source models on two safety benchmarks and one real-world task.

PrefIx: Understand and Adapt to User Preference in Human-Agent Interaction (2026.findings-acl)

Copied to clipboard

Jialin Li, Zhenhao Chen, Hanjun Luo, Hanan Salam

Challenge:	Current benchmarks evaluate task accuracy but overlook how agents interact . Preference-aware agents show 7.6% average UX improvement and 18.5% gain in preference alignment.
Approach:	They propose a configurable environment that evaluates both what agents accomplish and how they interact.
Outcome:	The proposed model improves performance and improves user experience by 7.6% and 18.5% respectively.

Safety Sidecar: Reflection-Driven Runtime Control for Safer Agents (2026.findings-acl)

Copied to clipboard

Wang Bin, Quan Jiazheng, Xingrui Yu, Hu Hansen, Yu Hao, Anjun Gao, Zhenglin Wan, Hui LI, Ivor Tsang

HiAgent: Hierarchical Working Memory Management for Solving Long-Horizon Agent Tasks with Large Language Model (2025.acl-long)

Challenge:	Existing safety controls fail to provide runtime intervention or cross-architecture portability for autonomous LLM agents.
Approach:	They propose a model-agnostic, plug-and-play module to provide arbitrary agent safety control and auditability.
Outcome:	The proposed module improves the secure-solution rate by 2.9–11.2 percentage points . it adds only 3.2s to end-to-end latency and a negligible average cost of 5.37 10-4 per scenario .

Copied to clipboard

Mengkang Hu, Tianxing Chen, Qiguang Chen, Yao Mu, Wenqi Shao, Ping Luo

When 20 Agents Fail to Sort: The Distributed Sorting Benchmark for Scalable Multi-Agent Systems (2026.findings-acl)

Challenge:	Existing approaches to optimize agent performance by incorporating entire historical action-observation pairs into LLMs are redundant in long-horizon tasks.
Approach:	They propose a framework that leverages subgoals as memory chunks to manage working memory of LLM-based agents hierarchically.
Outcome:	The proposed framework achieves a twofold increase in success rate and reduces the average number of steps required by 3.8.

Copied to clipboard

Xin Yang, Junhao Wang, Bintao Tang, Xuxin Cheng, Cao Liu, Ke Zeng, Wenyuan Jiang

Challenge:	MAS-BENCH isolates coordination under explicit communication constraints . CAMOC significantly improves coordination success and efficiency across backends .
Approach:	They propose a distributed-sorting benchmark that isolates coordination under explicit communication constraints.
Outcome:	MAS-BENCH improves coordination success and efficiency across backends . CAMOC significantly improves efficiency under shared-state interaction .

LLM-Based Human-Agent Collaboration and Interaction Systems: A Survey (2026.findings-acl)

Copied to clipboard

Henry Peng Zou, Wei-Chieh Huang, Yaozu Wu, Jizhou Guo, Yankai Chen, Chunyu Miao, Hoang H Nguyen, Yue Zhou, Weizhi Zhang, Liancheng Fang, Hanrong Zhang, Fangxin Wang, Pengfei Zhang, Langzhou He, Yangning Li, Dongyuan Li, Renhe Jiang, Philip S. Yu

ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and Feedback (2026.findings-acl)

Challenge:	Recent advances in large language models (LLMs) have sparked growing interest in building fully autonomous agents.
Approach:	They propose to integrate human-provided information, feedback, or control into the agent system to enhance system performance, reliability, and safety.
Outcome:	The proposed systems improve system performance, reliability, and safety by integrating human-provided information, feedback, or control into the agent system.

Copied to clipboard

Yutao Mou, Zhangchi Xue, Lijun Li, Peiyang Liu, Shikun Zhang, Wei Ye, Jing Shao